Engaging citizens in the development of a health system performance assessment framework: a case study in Ireland

Background The launch in 2017 of the Irish 10-year reform programme Sláintecare represents a key commitment in the future of the health system. An important component of the programme was the development of a health system performance assessment (HSPA) framework. In 2019, the Department of Health of Ireland (DoH) and Health Service Executive (HSE) commissioned the technical support of researchers to develop an outcome-oriented HSPA framework which should reflect the shared priorities of multiple stakeholders, including citizens. This study describes the method applied in the Irish context and reflects on the added value of using a citizen panel in the development of an HSPA framework. Methods A panel of 15 citizens was convened, recruited by a third-party company using a sampling strategy to achieve a balanced mix representing the Irish society. Panellists received lay-language preparatory materials before the meeting. Panellists used a three-colour scheme to signal the importance of performance measures. An exit questionnaire was administered to understand how participants experienced being part of the panel. The citizen panel was the first in a series of three panels towards the development of the HSPA framework, followed by panels including representatives of the DoH and HSE, and representatives from professional associations and special interest groups. Results The citizen panel generated 249 health performance measures ranging across 13 domains. Top-ranking domains to the citizen panel (people-centredness, coordination of care, and coverage) were less prioritized by the other panels; domains less prioritized by the citizen panel, such as accessibility, responsiveness, efficiency, and effectiveness, were of higher priority in the other panels. Citizen panellists shared a similar understanding of what a citizen panel involves and described their experience at the panel as enjoyable, interesting, and informative. Conclusions The priorities of the citizen panel were accounted for during all phases of developing the HSPA framework. This was possible by adopting an inclusive development process and by engaging citizens early on. Citizen engagement in HSPA development is essential for realizing citizen-driven healthcare system performance and generating trust and ownership in performance intelligence. Future research could expand the use of citizen panels to assess, monitor, and report on the performance of healthcare systems. Supplementary Information The online version contains supplementary material available at 10.1186/s12961-021-00798-8.


Output of this panel
All participants will receive a summary document on the citizen panel. In early 2020, this document will also be shared at a workshop with representatives of the Department of Health and other key stakeholders. The feedback from the citizen panel will be taken into consideration together with scientific evidence for making decisions about what to measure and report on in Ireland about the health system's performance.
Background: why is it important to know how the health system is doing?
There is a need for a public conversation about what we need to know about the health system in Ireland in order to make it work better.

Why does this matter in Ireland right now?
It is an important time to discuss what information policy-makers, health care providers and citizens would like to have about how the health system is doing (in other words, how it is performing) for a number of reasons. For example: • Ireland aims to improve the health care services people receive by its new programme, called 'Sláintecare'.
• At the moment, it is difficult to know how the health system is doing and where it needs to be changed or kept the same. This is partly because we are not yet collecting and publicly sharing the information people find important.
• Developing a method to collect the right information on the health system in Ireland has been prioritized and a number of activities, like this citizen panel, have been organized with the funding support of the European Commission to help the process.

Why is it important to know how the health system is doing (performing)?
We find four key reasons why it is important to know how the health system in Ireland is doing.
1. To identify things that need improvement, change or attention so that the health system will work better to meet the needs and expectations of patients and citizens.

2.
To be open and take responsibility on how the health system is doing.
3. To see over time how the health system is changing, since health needs and the expectations of citizens may also change over time.
4. To plan and organize the different activities of the health system so that different providers like nurses, general practitioners or hospital doctors will work better to care for patients.

The problem: why is it difficult to decide on what we want to know about the health system?
Collecting information on how the health system is doing is complex. This is because of a number of factors.
1. There are also things outside the health system that influence health One key challenge to collecting information on how the health system is doing, is deciding what to measure. This is partly because there are many things that take place outside of the health system that may influence a person's health and wellbeing. For example, the education system and the environment.

There is no "right way" to collect and share information about how the health system is doing
What type of information to collect and how to share this information needs to be decided in each country on its own. There is a lot that can be learned from other countries or from international organizations like the World Health Organization. However, in the end, it is up to Ireland to decide what information will be collected, used and shared on the Irish health system.

Access to good, trustworthy data is important but also difficult
Sometimes what we want to know is difficult to get because we do not have the right information available, or it is too expensive to get those data. Therefore, it is important to decide what is prioritized for measuring in the future. To judge how the health system is doing, information needs to be collected on a set of priority topics. The selection of such topics is an important decision. These topics are typically called domains. For each domain, information on more than one aspect of the health system could be collected.
We have summarized the most commonly used domains by other countries. We included a short explanation of what it means and an example of the type of information that can fall under that domain.

Health and well-being
The state of health and well-being reported by major themes related to health at large. For example, how long people are expected to live (life expectancy).

Equity
Delivering health care services that do not differ according to personal characteristics such as gender, race, ethnicity, geographical location or socioeconomic status. For example, the differences between counties in life expectancy.

Accessibility
The ability for individuals to easily reach health services in terms of location, time and convenience. For example, the number of individuals waiting for more than 120 days for a specific health care appointment.

Quality of care
The degree to which health services increase the likelihood of desired health outcomes and are consistent with current professional knowledge. For example, the number of births by caesarean section instead of natural deliveries.

Social and financial risk protection
The extent to which people can pay for the services needed without undue hardship. For example, how much people pay out-of-pocket payments for health care services or medications.

Coverage
The extent to which people in need of health care actually receive it. For example, the number of children immunized (vaccinated) to prevent childhood diseases.

Safety
Health care that minimizes risks and harm to patients, such as avoiding accidents, hospital complications, preventable injuries and reducing medical errors. For example, the number of slip and falls that occur in long-term care facilities.

Responsiveness
The ability of the health system to meet the needs and expectations of citizens. For example, the overall satisfaction of users with the health care system.

Domain
What does it mean? How do we measure it?

Efficiency
The delivery of health care in ways that maximizes resource use and avoids waste. For example, the number of generic (no-name) drugs rather than brand names prescribed by doctors.

Effectiveness
Health care services delivered based on scientific knowledge and the best available evidence. For example, the number of people that are go back to the hospital within 30 days of being hospitalized (readmissions).

People-centred
Care that takes into account the preferences and needs of citizens. For example, doctors taking into consideration the treatment wishes or preferences of patients.

Continuity of care
The delivery of care over time so a patient experiences a continuous caring relationship with their regular health care provider and undisrupted treatment. For example, a doctor's follow-up on a patient's medication for people with chronic conditions.

Coordination of care
The deliberate organization of a patient's care between two or more health care providers (e.g. a specialist in a hospital and their regular general practitioner) to ensure the best delivery of health care services. For example, the time taken for a patient's to be referred back to their general practitioner after being in hospital.